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Abstract — Current evolutions of language testing have led to integrating computers in FSP assessments both in 
oral and written communicative tasks. This paper deals with two main issues: learners’ expectations about the 
types of questions in FSP computer based assessments and the relation with their own experience. This paper 
describes the experience of 23 freshmen French students enrolled in the Universitat Politecnica de Valencia 
who took a computer based version of a FSP test. Their attitudes were observed and annotated. The learners 
expressed their difficulties in taking a FSP computer delivered test. The paper suggests that the dramatic 
change between general French and French for Specific Purposes may imply significant differences in the 
students to the new computer context due to the changes in vocabulary, register and language use as well as 
computer ergonomics. The paper concludes: firstly, that basic notions of certain specialized forms and 
vocabulary should be introduced in high school; secondly, that FSP should tend to be more communicative 
and that FSP teaching should be more reflective and communicative than it may be in colleges at the moment; 
thirdly, that if computer based tests may be a valuable tool in FSP testing (Garcia Laborda et al, 2010), item 
choice and prompts should be carefully considered. The analysis hereby presented is contextualized in 
computer based language testing and is a relevant part of the OPENPAU (MINECO FFI2011-22442) project. 

Index Terms — testing, second language, computers, French 


I. Introduction 

Current evolutions of language testing have led to integrating computers in FSP assessments both in oral and written 
communicative tasks (Roever, 2001; Chapelle and Douglas, 2006; Garcia Laborda, 2007) due to factors such as 
economy and task versatility. Overall, computers seem to fit perfectly with the new demands of modern and academic 
oriented higher education test constructs (Chapelle et al. , 2008; Garcia Laborda, 2010), additionally the Web 2.0 is 
beginning to be used for communicative tasks in the latest developments of computer assisted language testing (Vurdien, 
2013; Zou, Xing, Wang, Sun & Xiang, 2013; Dudeney & Hockly, 2012). 

This paper deals with two main issues: learners’ expectations about the types of questions in French for Specific 
Purposes (FSP) computer based assessments and the relation with their own experience. One of the premises stated by 
this tentative research in French is that although the students’ perceptions towards computer based language testing is 
that bothpen-and-paper and computer based testing processes (Newton, Acres & Bruce, 2013; Bayazit & Askar, 2012; 
Inuwa, Taranikanti, Al-Rawahy, & Habbal, 2012; Titus, 2012) are completely different, in fact apart from subtle 
differences such as the incorporation of mini clips and the improvement of sound quality, differences are usually related 
to the delivery context (Wauters, Desmet, & Van 2010; de Siqueira, Saez, A., Sevilla-Pavon, & Gimeno-Sanz, 201); Jin, 
2011); Secolsky, & Denison, 2011), feedback performance (gimeno Sanz & De Siqueira, 2009; Jordan & Mitchell, 
2009), report production and so. However, student’s task perception in language testing (and many other educational 
aspects) usually comprises part the attitudes construct (Afshari, Bakar, Luan, & Siraj, 2012; Deutsch. Herrmann, Frese, 
& Sandholzer, 2012); Jamil, Tariq, & Shami, (2012); van, Eggen, Timmers, & Veldkamp, 2012); Timmers, Braber-van, 
& van, 2013) and also shapes motivation towards educational changes (such as the acceptance of computer based 
language testing) (Barkaoui, Brooks, Swain, & Lapkin, 2013; Chong, & Lee, 2012; Deutsch, Herrmann, Frese, & 
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Sandholzer, 2012;. Weigle, 2013) Therefore, finding out what the students may think about the computer based tasks 
seems to be a good indicator for prospective motivation and, in the long run, acceptance and preference. For example, 
students with two to three years of practice in low stakes foreign language tests may be more adapted and open to take 
high stakes tests that can have great impact in the professional and academic career by allowing them to achieve high in 
the university entrance requirement (for instance, in the TOEFL), and access to international universities worldwide. 

In relation to their own experiences obtained through computer assisted language tests (CALT), students may have to 
work with computer for uses far beyond their education in shapes that cannot be predicted currently. Therefore, learning 
to interact with computers in very compromising situations such as a test may reinforce their mental strength and 
liberate them from undesired level of anxiety. Besides, students today seem to enjoy interacting with computers and 
Garcia Laborda et al. (2010a) have verified that operating with computers in language tests can lead to pleasing states of 
mind currently associated with the “flow effect” like in gaming. 

n. Experimental Phase 

This research addressed two main issues: learners’ expectations about the types of questions in French for specific 
purposes) computer based assessments (namely. Architecture and the relation with their own experience. 23 freshmen 
French students enrolled in the Universitat Politecnica de Valencia who took a 7 item computer based version of a FSP 
test with a format similar to the one found in the Spanish University Entrance examination with a speaking online task. 
The test gave some feedback through two one hour follow-up focus group meetings. Besides, their attitudes were 
observed and annotated while taking the computer based test. 

Special considerations were assumed in relation to the following aspects 

1 ) Applied computer literacy. Although students may be well familiarized with the latest trends in interface and 
commercial applications, they may also feel awkward when using educational applications especially for testing and 
even more for language since most of the 23 students had never taken an online language test and hardly worked with 
language computer applications. 

2) Testing conditions. Although students knew that the test would have little effect in their grades, testing is always 
stressful especially when there is no previous experience. 

3) Software Beta version. Beta versions can be problematic at times and the final result of test can be jeopardized by 
malfunctions. 

4) Utility. Even considering that students performed as in a real test, they may encounter little utility in the results 
since no strong feedback can be obtained from trial tests and thus underachieve in the test. 

Observations While Taking the Test 

The researchers also considered observing the realization of tasks, their fluency in approaching the tasks and finally 
the kind of interaction with the test administrators: 

1 ) The realization of tasks. Students seemed to be familiarized with the basic visual ergonomics of the testing 
platform after a short presentation. Apparently the intuitive interface (Described by Garcia Laborda et al., 2010b), its 
simplicity and visual iconicity facilitated their navigation. As verified through the test analysis, students did not 
evidence problem with timing and the use of tutorials while taking the test was almost null. Overall, the researchers 
observed that students tended to follow the test order straightforward with very few exceptions. 

2) Fluency in approaching the tasks. Students performed in the multiple choice tasks similarly to how they would 
have performed in normal testing situations. Thus, our interest was placed in the writing and speaking sections. The 
researchers noticed that students had few problems with both types of tasks. The average time spent on these tasks, as 
mentioned above, was adequate so the team considered that the students had adapted properly to the testing platform. 

3) Interaction with the test administrators. Students demanded little attention and asked very few clarifications while 
taking the test. Apparently, the previous experience the students in the study had with the computers was sufficient to 
allow them approach the test satisfactorily. 

III. Results from Focus Groups Meetings 

The main information for this paper was obtained through qualitative techniques. First, observations have been 
presented. Then we preferred focus groups meetings because “social investigation can be enhanced by employing the 
group interview technique . . . [since it] will provide data on group interaction, on realities as defined in a group context, 
and on interpretations of events that reflect group input” (Frey & Fontana, 1993, p. 20-21). In this case the researchers 
believed that data obtained from individual interviews would be repetitive. We considered that individual interviews 
would bring light into personal processes such as strategies but when a topic is as new as this one for the students, it 
seems very natural not to get all the intended information and individual interviews can be biased by the interviewer. 
Denzin and Lincoln (1998) state that focus groups bring different “level[s] of data gathering perspective on the research 
problem” (p. 53-54) that was considered difficult to obtain through face-to-face interviews. In the focus groups the 
following data was obtained: 

a) First reactions. Students acknowledged the benefits of computers in language testing in relation to the following 
aspects: (1) computer ergonomics, (2) types of questions, (3) enriched images for listening comprehension speaking 
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activities, (4) appropriateness of tasks. They also mentioned some drawbacks such ID recognition (due to the limited 
shape of the identity database at that time). Although in general students were happy with the given time, some two had 
to interrupt the test for different reasons and, obviously, since testing platforms are thought to be use in test conditions, 
they could not stop their timers so they had problems with time which had an effect on communicative tasks. 

b) Communicative tasks. At the beginning, some students were somehow shocked by their interaction and seven 
students had to do the speaking task within the given time. Although debatable in light of more recent research into task 
design, in this research the competence model by Dobrovolsky’s (1996) was followed to address two main aspects: task 
organization and whether pragmatic competence has a role in French for specific purposes computer based testing. 


Communicative competence 


Language 

competence 



Strategic 

competence 


Psychophysiological 

mechanisms 


Pragmatic 

competence 


Grammatical Textual 

competence competence 

Vocabulary/ \ Cohesion Rhetorical 
j \ organization 

/ Syntax 

Morphology | 


Illocutionary 

competence 


Sociolinguistic 

competence 

Dialect / Cutural 
/ references 

Register 


Functional 

abilities 


Phonology 

Figure 1 . Model of communicative competence (O’Grady & Dobrovolsky, 1 996) 


In relation to task organization, students seem to follow the linear order established by the test administrators and 
found little difficulties with the instructions. Students though that communication was rather limited but not any 
different from any other test. In fact, they felt at ease working face to face with videos and some students even preferred 
to do the speaking tasks in a computer instead of a human teacher. They mentioned the less afraid of making mistakes 
in this mode than when they considered that a human rater would position in relation to the testee. They also mentioned 
that the miniclips helped them to respond to the questions by providing ideas that were not explicit and direct responses 
to the questions. They also mentioned that questions extremely related to their professional field were too difficult but 
felt good because those questions were linked to more personal questions that let broader ground for personal responses. 
They also mentioned that they paid more attention to form than content and that probably prevented them from giving 
more risky responses to the questions. In relation to pragmatic competence, they observed that the interaction was too 
“static” and that they would have felt “different” in a person-to-person conversation. They thought that the method was 
good for descriptions and short answers but not very communicative. On the other hand, since their understanding that 
languages for specific languages tend to be in form of monologues (presentations, for instance) they believed that this 
computer based testing system would be acceptable for situations in which the speaker is not expected to interact 
frequently with the audience. 

a) The importance of training and preparation. In the students’ opinion, even though computer tests can be relevant 
to their professional skills, computer tests need (or should) be done in two or three sessions. In that case, each section 
could be enlarged especially the listening and speaking sections. They also think that if a test of this kind is to be 
implemented at any level of education, it needs to be prepared in class with certain amount of time and it could also be 
helpful to have some media devices like some of them had in their high school textbooks such as CD Roms, mobile 
devices or specific websites (Gimenez Lopez et al. (2010). 


IV. Conclusions 

Computer based language tests can be beneficial for both test administrators and test takers (Garcia Laborda, 2012; 
Garcia Laborda, Bakieva, Gonzalez-Such, & Pavon, 2010). However, these tests require training (Barkaoui, K., Brooks, 
Swain, & Lapkin, 2013) which sometimes is not available for the students. In relation to communication, speaking tasks 
in computer based language tests seem not very communicative at this point. Although some researchers have 
considered the possibility of semi-directed speaking tasks (Garcia Laborda, 2010) have suggested some aspects of 
interaction will have to be addressed in the future. 

In this experience, the learners expressed their difficulties in taking a FSP computer delivered test. They also 
mentioned that there is a great gap in communicative tasks between general French and French for specific purposes 
that may imply significant differences in their own adaptation to the new computer context due to the changes in 
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vocabulary, register and language use as well as computer ergonomics. Further research may need to address the 
following aspects: first, the introduction of basic notions of certain specialized forms and vocabulary while in high 
school; second. FSP should tend to be more communicative; third, FSP teaching should be more reflective and 
communicative than it may be in colleges at the moment; fourth, if computer based tests may be a valuable tool in FSP 
testing (Garcia Laborda et al, 2010) in the future, item choice and prompts should be carefully considered. The analysis 
hereby presented is contextualized in computer based language testing and is a relevant part of the OPENPAU project 
(MINECOFFI20 11-22442) project. 
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